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INTRODUCTION 


Psychologically-based  treatments  and  cognitive  behavioral  therapy  (CBT)  interventions  have 
been  shown  to  be  effective  in  alleviating  symptoms  of  Post-Traumatic  Stress  Disorder  (PTSD) 
and  related  psychological  health  problem  in  Veterans  and  military  personnel  who  suffer  from 
these  problems.  To  meet  the  increased  service  of  Veterans  with  PTSD,  new  training  methods 
need  to  be  developed  which  are:  1)  evidence-based,  and  2)  effective  in  modifying  and 
sustaining  changes  in  provider  behavior.  Methods  of  training/implementation  must  also  be 
scalable,  and  feasible  for  delivery  to  large  numbers  of  providers  in  cost-effective  ways. 
Internet-based  training  is  a  promising  new  approach  for  meeting  this  need,  but  has  received 
little  systematic  evaluation  to  date.  Noting  the  urgency  and  high  priority  of  this  issue,  Fairburn 
and  Cooper  (201 1 )  have  advocated  strongly  for  the  development  of  novel,  internet-based 
training  methods  and  innovative  research  designs  to  test  the  effectiveness  of  these  new  training 
methods.  Our  current  program  of  research  is  aimed  to  address  these  needs. 


The  broad  objective  of  our  research  is  to  design,  implement  and  evaluate  scalable  and  cost- 
effective  new  methods  for  training  of  mental  health  clinicians  providing  treatment  services  to 
veterans  with  PTSD.  The  randomized  controlled  trial  (RCT)  design  is  briefly  as  follows:  eligible 
clinicians  in  the  community  and  VHA  will  be  randomly  assigned  in  equal  numbers  to  three 
parallel  intervention  condition:  a)  Web-based  training  plus  web-centered  supervision;  b)  Web- 
based  training  alone;  and  c)  Training-as-usual  control  group.  An  equal  number  of  clinician 
trainees  from  VHA  (N=219)  and  the  community  (N=219)  will  be  recruited  and  enrolled  in  the 
study  over  an  18-month  period  according  to  a  randomized,  stratified  24-week  design. 
Comprehensive  assessments  will  be  performed  at  baseline  (TO),  completion  of  training  (T1), 
and  at  3  month  follow-up  (T2).  Participants  randomized  to  the  consultation  condition  will  be 
exposed  to  a  newly  developed  web-centered  form  of  learning  consultation.  Measures  of 
compliance  and  completion  will  assess  adherence  to  protocol.  Training  effectiveness  will  be 
evaluated  by  means  of  a  combination  of  objective  (SPE)  and  self-report  measures. 


The  primary  and  secondary  aims  of  the  study  are  as  follows: 

Primary  Aim:  To  compare  an  enhanced,  internet-based  training  intervention  combined  with 
novel  web-centered  supervision,  internet-based  training  intervention  without  web-centered 
supervision  and  a  wait-list  control  with  regard  to  improvements  in  two  CBT-based  skill  areas 
(behavioral  task  assignment  and  case  conceptualization).  We  hypothesize  that  enhanced, 
internet-based  training  in  conjunction  with  web-centered  supervision  will  result  in  superior  skills 
acquisition  compared  to  internet  training  alone  and  that  internet  training  alone  will  result  in 
superior  CBT  skills  than  wait-list  control. 

Secondary  Aim  #1:  To  compare  improvements  in  knowledge  and  attitudes  following  internet- 
based  training  with  or  without  web-centered  supervision  and  the  control.  We  hypothesize  that 
web-centered  supervision  will  lead  to  greater  improvements  in  CBT  knowledge  and  perceived 
self-efficacy  compared  to  internet-based  training  without  supervision  or  a  written  training-only 
condition.  We  hypothesize  similarly  that  internet-based  training  will  be  associated  with  improved 
outcomes  in  CBT  knowledge  and  attitudes  compared  to  a  written  training-only  condition. 

Secondary  Aim  #2:  To  compare  improvements  in  skills  acquisition  in  knowledge  and  attitudes 
following  training  in  clinicians  recruited  from  VHA  mental  health  treatment  settings  compared  to 
those  providing  services  in  civilian  community-based  clinics.  We  hypothesize  that  comparable 
improvements  will  be  achieved  in  the  trainees  from  civilian  community-based  clinics  compared 
to  clinicians  recruited  from  VHA  centers. 
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Secondary  Aim  #3:  To  determine  whether  clinician  implementation  of  skills  assessed  by 
means  of  a  novel,  objective  measure  of  skills  (i.e.,  standardized  patient).  We  hypothesize  that 
that  comparable  improvements  will  be  achieved  in  the  trainees  from  civilian  community-based 
clinics  compared  to  clinicians  recruited  from  VHA  centers. 

Secondary  Aim  #4:  To  assess  the  relative  efficiency  of  training,  as  measured  by  total  time 
required  for  training  in  each  condition,  in  addition  to  self-reported  level  of  burden  for  clinicians. 
We  hypothesize  that  internet-based  training  with  or  without  web-centered  supervision  will  be 
associated  with  increased  time  investment  and  burden  relative  to  training-as-usual,  but  that 
absolute  levels  of  burden  will  be  low  in  the  web  training  conditions. 


Our  study  will  be  the  first  of  its  kind  to  systematically  compare  web-based  training  interventions 
across  treatment  settings  and  provider  groups  (VHA  vs  Non-VHA).The  study  will  also  be  unique 
in:  1)  developing  and  testing  of  new  web-enhanced  training  modules  and  a  novel  web-centered 
supervision  model  recently  proposed  by  Fairburn  &  Cooper  (2011);  2)  development  and 
implementation  of  a  new  patient-reported  measure  of  clinician  skill  and  competency;  and  3) 
assessment  of  post-training  maintenance  of  skills  beyond  the  training  period.  Our  focus  on 
broad-based,  generic  CBT  skills  rather  than  more  narrowly  focused  protocol-based  skills  is 
another  innovative  aspect  of  our  proposed  study.  Finally,  the  use  of  standardized  patient 
methodology  for  assessing  outcomes  of  training,  and  planned  comparisons  with  self-report  and 
knowledge-based  assessment,  is  another  novel  feature  of  our  proposed  study. 


If  successful,  the  study  will  promote  a  better  standard  of  care  for  psychological  health  of 
Veterans  and  their  families  by  evaluating  technical  feasibility  of  two  training  models  in  evidence- 
based  skills  for  PTSD  treatment  providers  and  measuring  their  outcomes  and  effectiveness.  If 
successful,  the  study  will  provide  experimental  support  for  broad  implementation  of  these 
enhanced  new  training  methods  across  a  variety  of  treatment  settings. 


BODY 

This  section  of  the  report  will  describe  the  research  accomplishments  associated  with  each  task 
outlined  in  the  approved  Statement  of  Work  (SOW).  As  stated  in  our  SOW,  the  major  tasks  to 
be  accomplished  in  year  1  were: 

1.  Develop  and  Finalize  Study  Protocol  and  Measures 

A.  Eligibility  criteria,  exclusion  criteria,  screening  protocol 

i.  During  the  first  year  of  this  project,  the  team  finalized  the  eligibility  criteria,  exclusion 
criteria  and  screening  protocol. 

B.  Sample  frame,  web  contact  methods,  email  lists  identified 

i.  During  the  first  year  of  the  project,  the  team  has  identified  potential  participants  as 
well  as  identified  exisiting  email  lists  of  potential  participants  through  collaboration 
with  community  based  groups. 

a.  Email  lists  include:  VHA  mental  health  clinicians  that  have  participated  in  the 
“clinical  training  program”  at  the  National  Center  for  PTSD  in  the  past,  VHA 
mental  health  clinicians  who  have  not  participated  in  the  “clinical  training 
program”  at  the  NCPTSD,  team  leaders  at  the  Vet  Centers  in  the  United 
States,  and  a  registry  of  community  practitioners  who  have  agreet  to 
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volunteering  pro  bono  to  serve  the  needs  of  veterans  (Give  an  Hour).  A  large 
number  (6800)  community  clinicians  participate  in  this  registry  and  our  study 
has  arranged  access  to  this  recruitment  pool. 

b.  Postcards  have  been  created  to  be  disseminated  at  conferences  and  other 
mental  health  events  to  enhance  recruitment  activities  in  addition  to  the  pre¬ 
defined  email  lists. 

C.  Standardized  Patient  Interviews  scripted  and  pre-tested 

i.  A  standardized  patient  interview  script  has  been  created  for  assessing  clinicial  skills 
prior  to  and  following  the  training.  This  is  being  pre-tested  currently  prior  to 
finalization.  A  transcription  service  has  been  identified  and  contracted  for  creating 
transciptions  from  all  standardized  patient  interviews.  Actors  and  interviewers  have 
been  identified  for  the  study. 

D.  Knowledge  and  attitude  questionnaires  have  been  developed  for  the  study,  which  are 
currently  undergoing  pre-testing.  Other  measures  are  being  finalized  based  on  findings 
from  the  first  study  (Ruzek  et  al.,  2012). 

E.  Consent  form  drafted,  human  subjects  protocol  finalized 

i.  During  the  first  year  of  this  project,  the  web  based  consent  form  was  created.  The 
consent  form  and  protocol  has  been  approved  by  both  NERI  and  the  Stanford  IRBs. 
The  terms  of  the  study  will  be  clearly  defined  with  an  “accept”  or  “decline” 
participation  button.  Those  who  accept  will  progress  to  the  next  phase  of  the  study. 
Those  who  decline  will  be  thanked  Ifor  their  time.  All  consent  activities  will  occur 
through  a  secure  web  site. 

F.  NERI/VHA  IRB  approvals  and  USAMRMC  HRPO  human  subject  protocol  approval 

i.  The  Protocol,  Recruitment  materials,  Informed  Consent  documents,  applicable 
applications  have  been  approved  by  the  New  England  Research  Institutes,  Inc 
(NERI)  IRB  on  May  6,  2013.  Revised  Consent  forms  (version  4)  were  approved  by 
the  NERI  IRB  on  June  20,  2013  to  comply  with  the  DoD  requirements.  The  protocol 
was  reviewed  by  the  US  Army  Medical  Research  and  Materiel  Command 
(USAMRMC),  Office  of  Research  Protections  (ORP),  Human  Research  Protection 
Office  (HRPO)  and  found  to  comply  with  applicable  DOD,  US  Army,  and  USAMRMC 
human  subjects  protection  requirements.  This  approval  was  dated  July  2,  2013. 


2.  Develop  and  Finalize  Web-based,  CBT  Training  Materials 

A.  Develop  and  pre-test  CBT  instructional  modules  and  materials 

i.  During  the  first  year,  the  content  development  modules  have  been  desgined, 

developed  and  completed.  Module  content  for  the  video  clip  development  has  been 
finalized,  including  filming  and  final  cut  review.  Web  programming  of  content  is  on 
schedule  for  completion  in  October,  2013. 

B.  Develop  case  material  and  demonstrations 

i.  During  the  first  year  of  this  project,  the  team  developed  multiple  case  portrayalsfor 
use  in  the  video  simulation  interviews.  These  are  currently  being  pre-tested  prior  to 
finalization  of  the  measure. 

C.  Prepare  web-based  supervision  manual  and  materials 
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i.  During  the  first  year  of  this  project,  the  team  created  and  finalized  a  manual  and 
materials  for  the  web-based  supervision. 

D.  Training  content  complete,  reviewed  by  CBT  expert  consultants 

i.  A  team  of  carefully  selected  expert  content  consultants,  including  Christopher 
Fairburn,  MD,  Amy  Naugle,  Ph.D.,  Gareth  Holmen,  Ph.D.,  and  Brett  T.  Litz,  Ph.D., 
was  assembled.  These  individuals  were  invited  to  serve  as  special  consultants  to 
the  project  based  upon  their  extensive  experience  and  knowledge  in  these  content 
areas. 

E.  Pilot  test  all  study  procedures  and  materials  (prior  to  programming) 

i.  Pilot  testing  of  the  knowledge  items  has  been  initiated,  with  >  50  completed  surveys 
collected  and  both  qualitative  and  quantitative  data  collected.  A  combination  of 
blueprinting  and  concept  mapping  approaches  were  used  to  identify  key  concepts 
and  to  generate  specific  knowledge  assessment  items.  These  are  currently  being 
pre-tested  according  to  our  study  protocol  and  final  items  will  be  available  at 
randomization. 


3.  Develop.  Pre-Test  and  Finalize  Web  Site  and  Instructional  Program 

A.  VHA  web  host  programmers  provide  specification  and  guidance  to  web  programmers 
and  database  programmers 

i.  The  team  has  collaborated  with  the  VHA  NCPTSD  to  determine  integration 

requirements  for  ultimately  hosting  the  of  the  programmed  in  the  construction  of  our 
web-based  training  program  on  an  NCPTSD  server.  There  are  no  anticipated 
issues  with  hosting  the  program  with  the  National  Center  for  PTSD.  The  site  will  be 
compliant  with  and  fully  tested  for  508  compliance  requirements. 

B.  PTSD  training  material  completed.  All  web  training  modules  have  been  completed  and 
are  being  beta  tested  currently. 

C.  Web  programming  specifications  completed 

i.  During  the  first  year  of  this  project,  the  web  programming  specifications  were 

identified,  content  was  finalized  and  web  programming  activities  were  initiated.  Web 
programming  including  all  interactive  elements  is  on  schedule  for  completion  in 
October. 

D.  Web  program  transferred  to  VHA  intranet  server 

i.  Web  programming  has  not  yet  been  transferred  to  the  VHA  server  as  planned.  This 
site  will  be  hosted  externally  until  the  study  recruitment  is  completed,  at  which  time  it 
will  be  loaded  and  tested  on  the  VHA  NCPTSD  server. 


4.  Develop  Data  Management  System 

A.  Flow  chart  all  study  steps,  web  data  collection  and  database  requirements 

i.  During  the  first  year  of  this  project,  a  design  was  created  to  show  the  steps  in  data 
collection  to  plan  for  programming  of  the  DM  system. 

B.  Develop  web-based  data  forms  for  all  research  measures 
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i.  During  the  first  year  of  this  project,  the  team  has  begun  the  creation  of  web-based 
data  forms. 

C.  Program  the  database  including  project  data  entry  forms 

i.  During  this  period,  the  team  conducted  initial  programming  activities  fo  forms  in 
preparation  for  the  pilot.  These  activities  will  be  ongoing  over  the  next  two  months, 
with  completion  by  launch  in  the  first  quarter  of  the  second  year  grant  cycle. 

D.  Program  web  usage  statistics  reports 

i.  An  analysis  plan  has  been  developed  for  selection  and  validation  of  knowledge 
items.  These  reports  will  be  beta  tested  prior  to  initiation  of  the  study. 

E.  Program  reports  to  monitor  project  operations  and  outcomes 

i.  The  team  at  NERI  will  be  using  an  electronic  case  report  from  systems,  eCOS,  to 
store  the  data  forms.  The  reports  will  be  created  to  assist  with  monitoring  project 
operations.  Plans  have  been  initiated  in  this  project  year,  and  implementation  will 
be  undertaken  in  the  next  project  year. 


5.  Major  Task  Milestone 

A.  Program  automatic  email  reminders/interaction  with  web  participants 

i.  During  the  first  year  of  this  project,  the  email  reminder  process  has  been  reviewed 
and  is  an  ongoing  part  of  the  data  management  activities. 

B.  Beta  test  all  programmed  pieces  and  interactivity  with  web  course 

i.  Beta  testing  will  be  completed  by  launch,  the  first  quarter  of  the  second  year  grant 
cycle.  This  will  be  conducted  to  ensure  that  there  are  no  interactivity  errors  with  the 
web  course  of  the  evaluation  materials. 


6.  Prepare  Standardized  Patient  Rating  Protocol 

A.  Develop  and  Pre-test  Standardized  Patient  (SP)  Hating  Guide 

i.  Standardized  patient  case  materials  have  been  developed  and  are  in  pre-testing 
phase. 

ii.  New  rating  scales  have  been  developed  and  are  being  pre-tested  currently. 

B.  Study  raters  and  actors  have  been  engaged  for  the  study 

C.  Training  proceedures  have  been  developed  and  pre-tested. 

KEY  RESEARCH  ACCOMPLISHMENTS 

•  Hiring  and  training  of  all  project  staff 

•  Establishment  of  ongoing  communication  system  and  decision-making  processes  with 
partnering  institution  (VA  Menlo  Park) 

•  Prepared  and  finalized  research  protocol 

•  Draft  knowledge  and  skills  assessment  measures  and  initiate  pilot  testing 
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•  Obtained  all  relevant  approvals  for  study  protocol,  informed  consent  forms,  and 
recruitment  materials 

•  Developed  online  training  materials 

•  Initiated  web  programming  activities. 

•  Completed  video  filming  and  initiated  editing  process. 

•  Developed  study  assessment  methods  and  materials 

•  Pre-testing  of  study  assessment  methods  currently  underway 

•  Finalized  Statistical  Analysis  Plan 

•  Electronic  Data  Capture,  Data  Management  System  being  developed. 

•  Pilot  testing  was  initiated 


During  the  next  performance  period,  year  2,  the  team  will  focus  on  the  following  activities: 

1 .  Complete  Recruitment  Activities:  Based  on  our  established  plan  for  recruitment,  the 
team  will  develop  a  data  system  for  screening,  consenting  and  tracking  study 
participants. 

2.  Develop  Data  Management  System:  The  team  will  continue  to  work  on  development  of 
web  based  data  forms  for  all  research  measures  and  programming  the  database  to 
include  project  data  entry  forms. 

3.  Launch  of  web-based  training  program:  Completion  of  web  programming  for  training 
program,  including  installation  of  all  video  elements  (currently  being  reviewed  and 
undergoing  final  edit).  Launch  of  site  for  study  is  planned  for  the  first  quarter  of  the 
second  year  grant  cycle. 

4.  Complete  planning  and  implementation  of  email  reminders/interaction  with  web 
participants:  Email  reminders  are  anticipated  to  be  programmed  by  ,  the  first  quarter  of 
the  second  year  grant  cycle. 

5.  Standardized  Patient  Rating  Scales:  Pre-testing  of  interviews  and  standardized  patient 
rating  scales  are  in  process  and  scheduled  for  completion  in  04,  2013. 

6.  CE/CME  Accredidation:  As  study  participants  cannot  be  compensated  for  their  time  with 
anything  of  monetary  value,  CE/CMEs  will  be  issued  for  completion  of  the  web-based 
training.  CE/CME  accreditation  and  related  activities  are  in  process. 

7.  Recruitment:  Recruitment  materials  have  been  developed  and  approved. 


To  date  there  have  been  no  risks  or  unanticipated  issues  associated  with  this  project  that  have 
impeded  its  performance. 


Personnel  receiving  salary  from  this  research  effort  are  Raymond  C.  Rosen,  Ph.D.  (Partnering 
PI),  Lisa  Marceau,  MPH  (Co  Investigator,  Director),  Anne  Stoddard,  Ph.D.  (Statistician  and  Co- 
Investigator),  Michelle  Pucillo,  MPH  (Project  Manager),  Gayatri  Ranganathan,  MS  (Statistician), 
Julia  Coleman  (Research  Associate),  Julia  Frieze  (Senior  Research  Assistant),  Ellen  Devlin 
(Data  Management  Programmer),  Francine  Lombardi  (Data  Manager). 
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REPORTABLE  OUTCOMES 

To  date,  there  are  no  reportable  outcomes. 


CONCLUSION 

This  training  program  will  focus  on  development  of  evidence-based  CBT  skills  to  improve  skills 
in  providers  treating  Veterans  and  active  duty  military  and  to  effectively  engage  patients  in  the 
treatment  process.  This  innovative  study  will  add  new  knowledge  to  our  understanding  of  skills 
dissemination  in  PTSD  provider  care.  We  will  test  the  hypotheses  of  the  study  in  a  rigorous, 
experimental  design,  and  will  assess  outcomes  of  new  web-based  training  modules  and 
consultation  methods.  This  study  will  provide  data  to  assist  researchers,  military  leaders,  and 
treatment  providers  to  better  understand  practical  and  theoretical  implications  for  future  training 
of  mental  health  providers  in  the  VAH  and  other  health  systems. 


All  major  milestones  were  met  in  the  first  1 1  months  of  grant  activity. 
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